Methods and systems for detecting pleural irregularities in medical images

ABSTRACT

Various methods and systems are provided for a medical imaging system. In one embodiment, a method includes acquiring a series of medical images of a lung, identifying a pleural line in each medical image of the series, evaluating the pleural line for irregularities in each medical image of the series, and outputting an annotated version of each medical image of the series, the annotated version including visual markers for healthy pleura and irregular pleura. In this way, an operator of the medical imaging system may be alerted to pleural irregularities during a scan.

FIELD

Embodiments of the subject matter disclosed herein relate to ultrasoundimaging.

BACKGROUND

An ultrasound imaging system typically includes an ultrasound probe thatis applied to a patient's body and a workstation or device that isoperably coupled to the probe. During a scan, the probe may becontrolled by an operator of the system and is configured to transmitand receive ultrasound signals that are processed into an ultrasoundimage by the workstation or device. The workstation or device may showthe ultrasound images as well as a plurality of user-selectable inputsthrough a display device. The operator or other user may interact withthe workstation or device to analyze the images displayed on and/orselect from the plurality of user-selectable inputs.

As one example, ultrasound imaging may be used for examining a patient'slungs due to an ease of use of the ultrasound imaging system at apoint-of-care and low cost relative to a chest x-ray or a chest computedtomography (CT) scan, for example. Further, the ultrasound imagingsystem does not expose the patient to radiation. Lung ultrasoundimaging, also termed lung sonography, includes interpreting ultrasoundartifacts for diagnostic purposes. The ultrasound artifacts includeA-lines, which are substantially parallel and horizontal repeating linescaused by oscillating sound waves at pleura of the lungs, and B-lines,which are substantially vertical “comet-tail” artifacts of hyperechoicechoes indicating various lung pathologies including the presence offluid.

BRIEF DESCRIPTION

This summary introduces concepts that are described in more detail inthe detailed description. It should not be used to identify essentialfeatures of the claimed subject matter, nor to limit the scope of theclaimed subject matter.

In one aspect, a method can include acquiring a series of medical imagesof a lung, identifying a pleural line in each medical image of theseries, evaluating the pleural line for irregularities in each medicalimage of the series, and outputting an annotated version of each medicalimage of the series, the annotated version including visual markers forhealthy pleura and irregular pleura.

It should be understood that the brief description above is provided tointroduce in simplified form a selection of concepts that are furtherdescribed in the detailed description. It is not meant to identify keyor essential features of the claimed subject matter, the scope of whichis defined uniquely by the claims that follow the detailed description.Furthermore, the claimed subject matter is not limited toimplementations that solve any disadvantages noted above or in any partof this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure will be better understood from reading thefollowing description of non-limiting embodiments, with reference to theattached drawings, wherein below:

FIG. 1 shows a block schematic diagram of an ultrasound imaging system,according to an embodiment;

FIG. 2 is a schematic diagram illustrating an image processing systemfor detecting and classifying abnormalities in medical images, accordingto embodiment;

FIG. 3 shows a flow chart of an example method for detecting andquantifying pleural irregularities in real-time during a lungultrasound, according to an embodiment;

FIG. 4 shows a first example annotated panoramic lung ultrasound imagethat may be output to a display, according to an embodiment;

FIG. 5 shows a second example annotated panoramic lung ultrasound imagethat may be output to a display, according to an embodiment;

FIGS. 6A and 6B show a first example sequence of display outputs thatmay occur during and following a lung ultrasound, according to anembodiment;

FIGS. 7A and 7B show a second example sequence of display outputs thatmay occur during and following a lung ultrasound, according to anembodiment; and

FIG. 8 shows an example three-dimensional lung model having an overlaidultrasound image, according to an embodiment.

DETAILED DESCRIPTION

Embodiments of the present disclosure will now be described, by way ofexample, with reference to the FIGS. 1-8, which relate to variousembodiments for automatically detecting pleural irregularities based onmedical imaging data acquired by an imaging system, such as theultrasound imaging system shown in FIG. 1. As the processes describedherein may be applied to pre-processed imaging data and/or to processedimages, the term “image” is generally used throughout the disclosure todenote both pre-processed and partially-processed image data (e.g.,pre-beamformed RF or I/Q data, pre-scan converted RF data) as well asfully processed images (e.g., scan converted and filtered images readyfor display). An example image processing system that may be used todetect the pleural irregularities is shown in FIG. 2. The imageprocessing system may employ image processing techniques and one or morealgorithms to detect the pleural irregularities and output an indicationof the detected pleural irregularities to an operator, such as accordingto the method of FIG. 3. Various visual markers or indicators may beused to distinguish healthy pleura from irregular pleura, such asillustrated in FIGS. 5 and 6. As depicted by the example display outputsin FIGS. 6A-7B, the pleural irregularities may be detected and scored inreal-time, and then the system may select a best-frame representationfor extended display. Further, the system may output informationregarding a potential diagnosis or detected pathologies to the display.In some examples, a three-dimensional lung model also may be generatedand output to the display, with the best-frame representation overlaidin an anatomically and spatially relevant position, such as shown inFIG. 8. In this way, pleural irregularities may be detected andquantified from ultrasound imaging data in real-time, decreasing a timeuntil a diagnosis can be made and decreasing both intra-operator andinter-operator variation.

Advantages that may be realized in the practice of some embodiments ofthe described systems and techniques are that inconsistencies in thedetection of pleural irregularities, particularly between differentoperators, may be decreased. This may be particularly advantageous forincreasing a detection accuracy of point-of-care ultrasound operators,who may have less training than ultrasound experts (e.g., sonographersor radiologists). For example, an emergency room physician, who may notreceive expert-level ultrasound training, may be more likely to overlookan irregularity or incorrectly identify a normal structure or an imagingartifact as an irregularity, which may increase a burden on a radiologydepartment for follow up scans and increase patient discomfort. Further,by decreasing follow up scans and a mental burden on the point-of-careultrasound operator, an amount of time until an accurate diagnosis ismade may be decreased. Further still, the described system andtechniques may include outputting a “best-frame” image of any lungpathology present by selecting an image showing a greatest occurrence ofpleural irregularities and outputting a suggested diagnosis, furtherreducing the mental burden on the ultrasound operator and/or otherclinicians involved in diagnosing and treating the patient beingscanned.

Although the systems and methods described below for evaluating medicalimages are discussed with reference to an ultrasound imaging system, itmay be noted that the methods described herein may be applied to aplurality of imaging systems (e.g., MRI, PET, x-ray, CT, or othersimilar systems).

Referring to FIG. 1, a schematic diagram of an ultrasound imaging system100 in accordance with an embodiment of the disclosure is shown.However, it may be understood that embodiments set forth herein may beimplemented using other types of medical imaging modalities (e.g.,magnetic resonance imaging, computed tomography, positron emissiontomography, and so on). The ultrasound imaging system 100 includes atransmit beamformer 101 and a transmitter 102 that drives elements(e.g., transducer elements) 104 within a transducer array, hereinreferred to as a probe 106, to emit pulsed ultrasonic signals (referredto herein as transmit pulses) into a body (not shown). According to anembodiment, the probe 106 may be a one-dimensional transducer arrayprobe. However, in some embodiments, the probe 106 may be atwo-dimensional matrix transducer array probe. The transducer elements104 may be comprised of a piezoelectric material. When a voltage isapplied to the piezoelectric material, the piezoelectric materialphysically expands and contracts, emitting an ultrasonic spherical wave.In this way, the transducer elements 104 may convert electronic transmitsignals into acoustic transmit beams.

After the elements 104 of the probe 106 emit pulsed ultrasonic signalsinto a body (of a patient), the pulsed ultrasonic signals areback-scattered from structures within an interior of the body, likeblood cells or muscular tissue, to produce echoes that return to theelements 104. The echoes are converted into electrical signals, orultrasound data, by the elements 104, and the electrical signals arereceived by a receiver 108. The electrical signals representing thereceived echoes are passed through a receive beamformer 110 thatperforms beamforming and outputs ultrasound data, which may be in theform of a radiofrequency (RF) signal. Additionally, the transducerelements 104 may produce one or more ultrasonic pulses to form one ormore transmit beams in accordance with the received echoes.

According to some embodiments, the probe 106 may contain electroniccircuitry to do all or part of the transmit beamforming and/or thereceive beamforming. For example, all or part of the transmit beamformer101, the transmitter 102, the receiver 108, and the receive beamformer110 may be positioned within the probe 106. The terms “scan” or“scanning” may also be used in this disclosure to refer to acquiringdata through the process of transmitting and receiving ultrasonicsignals. The term “data” may be used in this disclosure to refer to oneor more datasets acquired with an ultrasound imaging system.

A user interface 115 may be used to control operation of the ultrasoundimaging system 100, including to control the input of patient data(e.g., patient medical history), to change a scanning or displayparameter, to initiate a probe repolarization sequence, and the like.The user interface 115 may include one or more of a rotary element, amouse, a keyboard, a trackball, hard keys linked to specific actions,soft keys that may be configured to control different functions, and agraphical user interface displayed on a display device 118. In someembodiments, the display device 118 may include a touch-sensitivedisplay, and thus, the display device 118 may be included in the userinterface 115.

The ultrasound imaging system 100 also includes a processor 116 tocontrol the transmit beamformer 101, the transmitter 102, the receiver108, and the receive beamformer 110. The processor 116 is in electroniccommunication (e.g., communicatively connected) with the probe 106. Asused herein, the term “electronic communication” may be defined toinclude both wired and wireless communications. The processor 116 maycontrol the probe 106 to acquire data according to instructions storedon a memory of the processor and/or a memory 120. As one example, theprocessor 116 controls which of the elements 104 are active and theshape of a beam emitted from the probe 106. The processor 116 is also inelectronic communication with the display device 118, and the processor116 may process the data (e.g., ultrasound data) into images for displayon the display device 118. The processor 116 may include a centralprocessing unit (CPU), according to an embodiment. According to otherembodiments, the processor 116 may include other electronic componentscapable of carrying out processing functions, such as a digital signalprocessor, a field-programmable gate array (FPGA), or a graphic board.According to other embodiments, the processor 116 may include multipleelectronic components capable of carrying out processing functions. Forexample, the processor 116 may include two or more electronic componentsselected from a list of electronic components including: a centralprocessor, a digital signal processor, a field-programmable gate array,and a graphic board. According to another embodiment, the processor 116may also include a complex demodulator (not shown) that demodulates RFdata and generates raw data. In another embodiment, the demodulation canbe carried out earlier in the processing chain.

The processor 116 is adapted to perform one or more processingoperations according to a plurality of selectable ultrasound modalitieson the data. In one example, the data may be processed in real-timeduring a scanning session as the echo signals are received by receiver108 and transmitted to processor 116. For the purposes of thisdisclosure, the term “real-time” is defined to include a procedure thatis performed without any intentional delay (e.g., substantially at thetime of occurrence). For example, an embodiment may acquire images at areal-time rate of 7-20 frames/sec. The ultrasound imaging system 100 mayacquire two-dimensional (2D) data of one or more planes at asignificantly faster rate. However, it should be understood that thereal-time frame-rate may be dependent on a length (e.g., duration) oftime that it takes to acquire and/or process each frame of data fordisplay. Accordingly, when acquiring a relatively large amount of data,the real-time frame-rate may be slower. Thus, some embodiments may havereal-time frame-rates that are considerably faster than 20 frames/secwhile other embodiments may have real-time frame-rates slower than 7frames/sec.

In some embodiments, the data may be stored temporarily in a buffer (notshown) during a scanning session and processed in less than real-time ina live or off-line operation. Some embodiments of the disclosure mayinclude multiple processors (not shown) to handle the processing tasksthat are handled by the processor 116 according to the exemplaryembodiment described hereinabove. For example, a first processor may beutilized to demodulate and decimate the RF signal while a secondprocessor may be used to further process the data, for example, byaugmenting the data as described further herein, prior to displaying animage. It should be appreciated that other embodiments may use adifferent arrangement of processors.

The ultrasound imaging system 100 may continuously acquire data at aframe-rate of, for example, 10 Hz to 30 Hz (e.g., 10 to 30 frames persecond). Images generated from the data may be refreshed at a similarframe-rate on the display device 118. Other embodiments may acquire anddisplay data at different rates. For example, some embodiments mayacquire data at a frame-rate of less than 10 Hz or greater than 30 Hzdepending on the size of the frame and the intended application. Thememory 120 may store processed frames of acquired data. In an exemplaryembodiment, the memory 120 is of sufficient capacity to store at leastseveral seconds' worth of frames of ultrasound data. The frames of dataare stored in a manner to facilitate retrieval thereof according to itsorder or time of acquisition. The memory 120 may comprise any known datastorage medium.

In various embodiments of the present disclosure, data may be processedin different mode-related modules by the processor 116 (e.g., B-mode,Color Doppler, M-mode, Color M-mode, spectral Doppler, elastography,tissue velocity imaging, strain, strain rate, and the like) to form 2Dor three-dimensional (3D) images. When multiple images are obtained, theprocessor 116 may also be configured to stabilize or register theimages. For example, one or more modules may generate B-mode, colorDoppler, M-mode, color M-mode, color flow imaging, spectral Doppler,elastography, tissue velocity imaging (TVI), strain, strain rate, andthe like, and combinations thereof. As one example, the one or moremodules may process color Doppler data, which may include traditionalcolor flow Doppler, power Doppler, high-definition (HD) flow Doppler,and the like. The image lines and/or frames are stored in memory and mayinclude timing information indicating a time at which the image linesand/or frames were stored in memory. The modules may include, forexample, a scan conversion module to perform scan conversion operationsto convert the acquired images from beam space coordinates to displayspace coordinates. A video processor module may be provided that readsthe acquired images from a memory and displays an image in real-timewhile a procedure (e.g., ultrasound imaging) is being performed on apatient. The video processor module may include a separate image memory,and the ultrasound images may be written to the image memory in order tobe read and displayed by the display device 118.

Further, the components of the ultrasound imaging system 100 may becoupled to one another to form a single structure, may be separate butlocated within a common room, or may be remotely located with respect toone another. For example, one or more of the modules described hereinmay operate in a data server that has a distinct and remote locationwith respect to other components of the ultrasound imaging system 100,such as the probe 106 and the user interface 115. Optionally, theultrasound imaging system 100 may be a unitary system that is capable ofbeing moved (e.g., portably) from room to room. For example, theultrasound imaging system 100 may include wheels or may be transportedon a cart, or may comprise a handheld device.

For example, in various embodiments of the present disclosure, one ormore components of the ultrasound imaging system 100 may be included ina portable, handheld ultrasound imaging device. For example, the displaydevice 118 and the user interface 115 may be integrated into an exteriorsurface of the handheld ultrasound imaging device, which may furthercontain the processor 116 and the memory 120 therein. The probe 106 maycomprise a handheld probe in electronic communication with the handheldultrasound imaging device to collect raw ultrasound data. The transmitbeamformer 101, the transmitter 102, the receiver 108, and the receivebeamformer 110 may be included in the same or different portions of theultrasound imaging system 100. For example, the transmit beamformer 101,the transmitter 102, the receiver 108, and the receive beamformer 110may be included in the handheld ultrasound imaging device, the probe,and combinations thereof.

Referring to FIG. 2, an example medical image processing system 200 isshown. In some embodiments, the medical image processing system 200 isincorporated into a medical imaging system, such as an ultrasoundimaging system (e.g., the ultrasound imaging system 100 of FIG. 1), anMRI system, a CT system, a single-photon emission computed tomography(SPECT) system, etc. In some embodiments, at least a portion of themedical image processing system 200 is disposed at a device (e.g., anedge device or server) communicably coupled to the medical imagingsystem via wired and/or wireless connections. In some embodiments, themedical image processing system 200 is disposed at a separate device(e.g., a workstation) that can receive images from the medical imagingsystem or from a storage device that stores the images generated by themedical imaging system. The medical image processing system 200 maycomprise an image processor 231, a user input device 232, and a displaydevice 233. For example, the image processor 231 may beoperatively/communicatively coupled to the user input device 232 and thedisplay device 233.

The image processor 231 includes a processor 204 configured to executemachine-readable instructions stored in non-transitory memory 206. Theprocessor 204 may be single core or multi-core, and the programsexecuted by the processor 204 may be configured for parallel ordistributed processing. In some embodiments, the processor 204 mayoptionally include individual components that are distributed throughouttwo or more devices, which may be remotely located and/or configured forcoordinated processing. In some embodiments, one or more aspects of theprocessor 204 may be virtualized and executed by remotely-accessiblenetworked computing devices configured in a cloud computingconfiguration. In some embodiments, the processor 204 may include otherelectronic components capable of carrying out processing functions, suchas a digital signal processor, a field-programmable gate array (FPGA),or a graphics board. In some embodiments, the processor 204 may includemultiple electronic components capable of carrying out processingfunctions. For example, the processor 204 may include two or moreelectronic components selected from a plurality of possible electroniccomponents, including a central processor, a digital signal processor, afield-programmable gate array, and a graphics board. In still furtherembodiments, the processor 204 may be configured as a graphicalprocessing unit (GPU), including parallel computing architecture andparallel processing capabilities.

In the embodiment shown in FIG. 2, the non-transitory memory 206 storesa detection and quantification module 212 and medical image data 214.The detection and quantification module 212 includes one or morealgorithms to process input medical images from the medical image data214. Specifically, the detection and quantification module 212 mayidentify an anatomical feature within the medical image data 214 andanalyze the anatomical feature for irregularities or abnormalities. Forexample, the detection and quantification module 212 may include one ormore image recognition algorithms, shape or edge detection algorithms,gradient algorithms, and the like to process input medical images.Additionally or alternatively, the detection and quantification module212 may store instructions for implementing a neural network, such as aconvolutional neural network, for detecting and quantifying anatomicalirregularities captured in the medical image data 214. For example, thedetection and quantification module 212 may include trained and/oruntrained neural networks and may further include training routines, orparameters (e.g., weights and biases), associated with one or moreneural network models stored therein. In some embodiments, the detectionand quantification module 212 may evaluate the medical image data 214 asit is acquired in real-time. Additionally or alternatively, thedetection and quantification module 212 may evaluate the medical imagedata 214 offline, not in real-time.

As an example, when the medical image data 214 includes lung ultrasounddata, the identified anatomical feature may include pleura, which may beidentified by the detection and quantification module 212 based onpleural sliding via edge detection techniques and/or gradient changes.As will be elaborated wherein with respect to FIG. 3, once pleuralpositioning is detected in each frame, the detection and quantificationmodule may score the pleura according to several parameters, includinglocal dimness and vertical location, to determine which pleurallocations are irregular.

Optionally, the image processor 231 may be communicatively coupled to atraining module 210, which includes instructions for training one ormore of the machine learning models stored in the detection andquantification module 212. The training module 210 may includeinstructions that, when executed by a processor, cause the processor tobuild a model (e.g., a mathematical model) based on sample data to makepredictions or decisions regarding the detection and classification ofanatomical irregularities without the explicit programming of aconventional algorithm that does not utilize machine learning. In oneexample, the training module 210 includes instructions for receivingtraining data sets from the medical image data 214. The training datasets comprise sets of medical images, associated ground truthlabels/images, and associated model outputs for use in training one ormore of the machine learning models stored in the detection andquantification module 212. The training module 210 may receive medicalimages, associated ground truth labels/images, and associated modeloutputs for use in training the one or more machine learning models fromsources other than the medical image data 214, such as other imageprocessing systems, the cloud, etc. In some embodiments, one or moreaspects of the training module 210 may include remotely-accessiblenetworked storage devices configured in a cloud computing configuration.Further, in some embodiments, the training module 210 is included in thenon-transitory memory 206. Additionally or alternatively, in someembodiments, the training module 210 may be used to generate thedetection and quantification module 212 offline and remote from theimage processing system 200. In such embodiments, the training module210 may not be included in the image processing system 200 but maygenerate data stored in the image processing system 200. For example,the detection and quantification module 212 may be pre-trained with thetraining module 210 at a place of manufacture.

The non-transitory memory 206 further stores the medical image data 214.The medical image data 214 includes, for example, functional and/oranatomical images captured by an imaging modality, such as an ultrasoundimaging system, an MRI system, a CT system, a PET system, etc. As oneexample, the medical image data 214 may include ultrasound images, suchas lung ultrasound images. Further, the medical image data 214 mayinclude one or more of 2D images, 3D images, static single frame images,and multi-frame cine-loops (e.g., movies).

In some embodiments, the non-transitory memory 206 may includecomponents disposed at two or more devices, which may be remotelylocated and/or configured for coordinated processing. In someembodiments, one or more aspects of the non-transitory memory 206 mayinclude remotely-accessible networked storage devices in a cloudcomputing configuration. As one example, the non-transitory memory 206may be part of a picture archiving and communication system (PACS) thatis configured to store patient medical histories, imaging data, testresults, diagnosis information, management information, and/orscheduling information, for example.

The image processing system 200 may further include the user inputdevice 232. The user input device 232 may comprise one or more of atouchscreen, a keyboard, a mouse, a trackpad, a motion sensing camera,or other device configured to enable a user to interact with andmanipulate data stored within the image processor 231.

The display device 233 may include one or more display devices utilizingany type of display technology. In some embodiments, the display device233 may comprise a computer monitor and may display unprocessed images,processed images, parametric maps, and/or exam reports. The displaydevice 233 may be combined with the processor 204, the non-transitorymemory 206, and/or the user input device 232 in a shared enclosure ormay be a peripheral display device. The display device 233 may include amonitor, a touchscreen, a projector, or another type of display device,which may enable a user to view medical images and/or interact withvarious data stored in the non-transitory memory 206. In someembodiments, the display device 233 may be included in a smartphone, atablet, a smartwatch, or the like.

It may be understood that the medical image processing system 200 shownin FIG. 2 is one non-limiting embodiment of an image processing system,and other imaging processing systems may include more, fewer, ordifferent components without parting from the scope of this disclosure.Further, in some embodiments, at least portions of the medical imageprocessing system 200 may be included in the ultrasound imaging system100 of FIG. 1, or vice versa (e.g., at least portions of the ultrasoundimaging system 100 may be included in the medical image processingsystem 200).

As used herein, the terms “system” and “module” may include a hardwareand/or software system that operates to perform one or more functions.For example, a module or system may include or may be included in acomputer processor, controller, or other logic-based device thatperforms operations based on instructions stored on a tangible andnon-transitory computer readable storage medium, such as a computermemory. Alternatively, a module or system may include a hard-wireddevice that performs operations based on hard-wired logic of the device.Various modules or systems shown in the attached figures may representthe hardware that operates based on software or hardwired instructions,the software that directs hardware to perform the operations, or acombination thereof.

“Systems” or “modules” may include or represent hardware and associatedinstructions (e.g., software stored on a tangible and non-transitorycomputer readable storage medium, such as a computer hard drive, ROM,RAM, or the like) that perform one or more operations described herein.The hardware may include electronic circuits that include and/or areconnected to one or more logic-based devices, such as microprocessors,processors, controllers, or the like. These devices may be off-the-shelfdevices that are appropriately programmed or instructed to performoperations described herein from the instructions described above.Additionally or alternatively, one or more of these devices may behard-wired with logic circuits to perform these operations.

Next, FIG. 3 shows a flow chart of an example method 300 for identifyingand quantifying pleural irregularities in lung images of a patient. Inparticular, method 300 provides a workflow for expediting and increasingan accuracy of diagnosing the patient. Method 300 will be described forultrasound images acquired using an ultrasound imaging system, such asultrasound imaging system 100 of FIG. 1, although other ultrasoundimaging systems may be used. Further, method 300 may be adapted to otherimaging modalities. Method 300 may be implemented by one or more of theabove described systems, including the ultrasound imaging system 100 ofFIG. 1 and medical image processing system 200 of FIG. 2. As such,method 300 may be stored as executable instructions in non-transitorymemory, such as the memory 120 of FIG. 1 and/or the non-transitorymemory 206 of FIG. 2, and executed by a processor, such as the processor116 of FIG. 1 and/or the processor 204 of FIG. 2. Further, in someembodiments, method 300 is performed in real-time, as the ultrasoundimages are acquired, while in other embodiments, at least portions ofmethod 300 are performed offline, after the ultrasound images areacquired. For example, the processor may evaluate ultrasound images thatare stored in memory even while the ultrasound system is not activelybeing operated to acquire images. Further still, at least parts ofmethod 300 may be performed in parallel. For example, ultrasound datafor a second image may be acquired while a first ultrasound image isgenerated, ultrasound data for a third image may be acquired while thefirst ultrasound image is analyzed, and so on.

At 302, method 300 includes receiving a lung ultrasound protocolselection. The lung ultrasound protocol may be selected by an operator(e.g., user) of the ultrasound imaging system via a user interface(e.g., the user interface 115). As one example, the operator may selectthe lung ultrasound protocol from a plurality of possible ultrasoundprotocols using a drop-down menu or by selecting a virtual button.Alternatively, the system may automatically select the protocol based ondata received from an electronic health record (EHR) associated with thepatient. For example, the EHR may include previously performed exams,diagnoses, and current treatments, which may be used to select the lungultrasound protocol. Further, in some examples, the operator maymanually input and/or update parameters to use for the lung ultrasoundprotocol. The lung ultrasound protocol may be a system guided protocol,where the system guides the operator through the protocol step-by-step,or a user guided protocol, where the operator follows a lab-defined orself-defined protocol without the system enforcing a specific protocolor having prior knowledge of the protocol steps.

Further, the lung ultrasound protocol may include a plurality ofscanning sites (e.g., views), probe movements, and/or imaging modes thatare sequentially performed. For example, the lung ultrasound protocolmay include using real-time B-mode imaging with a convex, curvilinear,or linear ultrasound probe (e.g., the probe 106 of FIG. 1). In someexamples, the lung ultrasound protocol may further include using dynamicM-mode. The lung ultrasound protocol may include a longitudinal scan,wherein the probe is positioned perpendicular to the ribs, and/or anoblique scan, wherein the probe is positioned along intercostal spacesbetween ribs. Further still, in some examples, the lung ultrasoundprotocol may include a panoramic sweep, where the user sweeps theultrasound probe from the head downward, and multiple views from thesweep may be stitched together to provide anatomical and spatialrelationships. In some embodiments, the probe may include athree-dimensional probe sensor that may give width information an imagedlung, while a length of the sweep may indicate a length of the imagedlung.

At 304, method 300 includes acquiring ultrasound data with theultrasound probe by transmitting and receiving ultrasonic signalsaccording to the lung ultrasound protocol. Acquiring ultrasound dataaccording to the lung ultrasound protocol may include the systemdisplaying instructions on the user interface, for example, to guide theoperator through the acquisition of the designated scanning sites.Additionally or alternatively, the lung ultrasound protocol may includeinstructions for the ultrasound system to automatically acquire some orall of the data or perform other functions. For example, the lungultrasound protocol may include instructions for the user to move,rotate and/or tilt the ultrasound probe, as well as to automaticallyinitiate and/or terminate a scanning process and/or adjust imagingparameters of the ultrasound probe, such as ultrasound signaltransmission parameters, ultrasound signal receive parameters,ultrasound signal processing parameters, or ultrasound signal displayparameters. Further, the acquired ultrasound data include one or moreimage parameters calculated for each pixel or group of pixels (forexample, a group of pixels assigned the same parameter value) to bedisplayed, where the one or more calculated image parameters include,for example, one or more of an intensity, velocity, color flow velocity,texture, graininess, contractility, deformation, and rate of deformationvalue.

At 306, method 300 includes generating ultrasound images from theacquired ultrasound data. For example, the signal data acquired duringthe method at 304 is processed and analyzed by the processor in order toproduce an ultrasound image at a designated frame rate. The processormay include an image processing module that receives the signal data(e.g., image data) acquired at 304 and processes the received imagedata. For example, the image processing module may process theultrasound signals to generate slices or frames of ultrasoundinformation (e.g., ultrasound images) for displaying to the operator. Inone example, generating the image may include determining an intensityvalue for each pixel to be displayed based on the received image data(e.g., 2D or 3D ultrasound data). As such, the generated ultrasoundimages may be 2D or 3D depending on the mode of ultrasound being used(such as B-mode, M-mode, and the like). The ultrasound images will alsobe referred to herein as “frames” or “image frames.”

At 308, method 300 includes detecting a pleural position in eachultrasound image. In an aerated lung, the pleura, which form the outerboundary of the lung that lies against the chest wall, may provide thesubstantially only anatomical lung structure detectable by ultrasound.The pleura appear as a hyperechoic horizontal segment of brighter (e.g.,whiter) pixels in the ultrasound image, referred to as a pleural line,which moves synchronously with respiration in a phenomenon known aspleural sliding. The processor may utilize an analysis module, such asthe detection and quantification module 212 of FIG. 2, to identify thehorizontal (or diagonal, in some orientations) line that demonstratesthe most amount of pleural sliding in order to define the pleuralposition (and thus the pleural line) in each image. For example, theprocessor may evaluate consecutive frames and identify a horizontal (ordiagonal) area having a highest amount of local change betweenconsecutive frames, such as by summing absolute differences betweenframes, to identify the pleural sliding and thus, the pleural position.In frames without pleural sliding (e.g., between breaths), the locationof the pleura may be tracked up/down based on its known location fromthe previous frame.

Additionally or alternatively, detecting the pleural position mayinclude identifying lower and upper borders of the pleura based on abrightness change between pixels, such as by using edge detectiontechniques or gradient changes. For example, the processor may apply anedge detection algorithm that comprises one or more mathematical methodsfor identifying points (e.g., pixels) at which the image brightnesschanges sharply and/or has discontinuities to identify the lower andupper borders of the pleural line. As one example, the processor mayapply the edge detection algorithm to the area having the highest amountof local change. As another example, additionally or alternatively, agradient algorithm may identify a local maximum and/or minimum pixelbrightness at the pleural position to identify the lower and upperborders of the pleural line in each image.

At 310, method 300 includes detecting and quantifying irregularities ineach ultrasound image. For example, when the air content decreases dueto the presence of fluid in the lung, the ultrasound signal may bepartly reflected at deeper zones than the pleura, resulting in verticalreverberation artifacts known as B-lines. As the number of B-linesincreases, an air content of the lung decreases and a density of thelung increases due to scarring and/or accumulation of fluid in thepulmonary interstitial space, such as through lung consolidation. Whileone or two B-lines may not indicate a disease state, more than twoB-lines or confluent B-lines may indicate irregularities, such as lungconsolidation.

Thus, as one example, detecting and quantifying irregularities in eachultrasound image may include identifying B-lines in each ultrasoundimage. The processor may identify the B-lines as discrete verticalhyperechoic reverberation artifacts that move synchronously with pleuralsliding, extending from the pleural line to the bottom of the image. Theprocessor may further sum the number of B-lines found in each frame.

Detecting and quantifying the irregularities in each ultrasound imagefurther includes detecting and quantifying pleural irregularities. Forexample, the processor may evaluate each pixel of the identified pleura(e.g., within the upper and lower borders of the pleural line) tolocally characterize the pleura as either healthy or irregular viapre-determined scoring criteria. As an example, the processor mayevaluate each pixel of the pleural line to determine a jumpiness scoreand a dimness score for each pleural location in order to identifypositions of pleural irregularities. The jumpiness score evaluates avertical location of the pleural line at each horizontal location toidentify vertical gaps in the pleural line, with a greater vertical gapresulting in a higher jumpiness score. For example, the vertical gap mayrefer to a number of pixels vertically between the lower border (orupper border) of the pleural line at the given pixel location relativeto a neighboring pixel. The vertical gap between the upper or lowerborder of the pleural line at neighboring horizontal locations mayresult in the pleural line having a discontinuous or rough appearance,for example. The dimness score ranks the pleural pixel brightness (ordimness) at a particular horizontal location relative to its neighbors.As the local pixel brightness of the pleura decreases relative to itsneighbors (e.g., the pixel becomes more dim relative to its neighbors),the dimness score increases.

An irregularity score for each pixel along the pleural line in eachframe may be generated as a product of the jumpiness score and thedimness score and compared to a threshold score. The threshold score maybe a pre-determined value stored in memory that distinguishes irregularpleura associated with a disease state from normal, healthy pleura. Insome examples, the threshold score may be adjusted based on curated dataand using a support vector machine. If the irregularity score is greaterthan or equal to the threshold score, the pleura imaged in that pixellocation may be considered irregular. In contrast, if the irregularityscore is less than the threshold score, the pleura imaged in that pixellocation may not be considered irregular (e.g., may be considered normaland/or healthy). Although the pleura may be analyzed on a pixel-by-pixelbasis, a filter may be used to smooth the results. As a result, an areaof pixels having pre-determined dimensions may be grouped and identifiedas a location of irregularity (e.g., irregular pleura) responsive to amajority (e.g., greater than 50%) of the pixels within the group beingcharacterized as irregular pleura. In contrast, the area of pixels maybe identified as healthy responsive to the majority of the pixels withinthe group being characterized as healthy pleura.

At 314, method 300 includes outputting annotated ultrasound images to adisplay. For example, the ultrasound images may comprise the pixelparameter values (e.g., brightness values) calculated at 306, and anannotated version of each ultrasound image that comprises the pixelparameter values overlaid with visual indications (e.g., annotations)regarding B-lines, the pleural position, and/or pleural irregularitiesmay be output to the display in real-time. In some examples, the displayis included in the ultrasound imaging system, such as display device118. For example, B-lines may be highlighted with a solid vertical line,and the upper and/or lower border (e.g., boundary) of the pleural linemay be indicated with markers or traced (e.g., with a line). Further,outputting the annotated ultrasound images includes distinguishing thehealthy pleura from irregular pleura, as indicated at 316. As oneexample, healthy pleura may be visually indicated with a first marker,while irregular pleura may be visually indicated with a second marker.The first marker may be an annotation, such as a dot, square, bracket,line, or arrow, having a first characteristic (e.g., a firstcharacteristic shape and/or color), and the second marker may be anannotation having a second characteristic that is different than thefirst characteristic. As another example, tissue colorization may beused to distinguish healthy pleura from irregular pleura, such as bycolorizing the healthy pleura with a first color and colorizing theirregular pleura with a second, different color. Each annotatedultrasound image may be output in substantially real-time in thesequence acquired and at a designated display frame rate.

Turning briefly to FIG. 4, a first example annotated panoramic lungultrasound image 400 that may be output to a display is shown. The firstexample annotated panoramic lung ultrasound image 400 includes apanoramic lung ultrasound image 402, healthy markers 404, irregularmarkers 406, and a B-line indicator 408. The panoramic lung ultrasoundimage 402 shows seven distinct pleural lines: a first pleural line 410of a first rib space, a second pleural line 412 of a second rib space, athird pleural line 414 of a third rib space, a fourth pleural line 416if a fourth rib space, a fifth pleural line 418 of a fifth rib space, asixth pleural line 420 of a sixth rib space, and a seventh pleural line422 of a seventh rib space, each pleural line separated by a darkportion in the image where a rib is located. Both the healthy markers404 and the irregular markers 406 are positioned on an upper boundary ofeach pleural line in the example shown, with the healthy markers 404visually indicating locations having healthy pleura and the irregularmarkers 406 visually indicating locations having pleural irregularities(e.g., as determined at 310 of FIG. 3).

In the present example, the healthy markers 404 are white squares withblack outlines, while the irregular markers 406 are black squares withwhite outlines, although other shapes and colors are also possible.Thus, the healthy markers 404 and the irregular markers 406 have thesame shape and size but different coloration. Further, the B-lineindicator 408 is shown as a vertical line.

The first pleural line 410 does not have any pleural irregularities(e.g., all of the markers are the healthy markers 404). The number ofpleural irregularities generally increases from the fourth pleural line416 toward the seventh pleural line 422, with the seventh pleural line422 having the most irregular pleura (e.g., the greatest ratio ofirregular markers 406 to healthy markers 404 of the seven pleurallines). The seventh pleural line 422 also includes an identified B-line,as indicated by the B-line indicator 408. Thus, the annotated panoramiclung ultrasound image 400 shows increasing irregularities from left toright in the panoramic lung ultrasound image 402.

However, other markers are also possible to distinguish healthy pleurafrom irregular pleura. Turning now to FIG. 5, a second example annotatedpanoramic lung ultrasound image 500 that may be output to a display isshown. The second example annotated panoramic lung ultrasound image 500includes the same panoramic lung ultrasound image 402 as in FIG. 4, andthus, components of FIG. 5 that are the same as those in FIG. 4 arenumbered the same and will not be reintroduced. For example, theannotated panoramic lung ultrasound image 500 includes the same healthymarkers 404 positioned above each pleural line. However, the annotatedpanoramic lung ultrasound image 500 includes irregular markers 506,which are shown as brackets above areas having irregular pleura, insteadof the irregular markers 406. Thus, the irregular markers 506 have adifferent shape, color, and position than the healthy markers 404.

Returning to FIG. 3, at 318, it is determined if the acquisition isfinished. For example, the acquisition may be considered finished whenultrasound data is acquired for all of the views and/or imaging modesprogrammed in the lung ultrasound protocol and the ultrasound probe isno longer actively transmitting and receiving ultrasonic signals.Additionally or alternatively, the acquisition may be finishedresponsive to the processor receiving an “end protocol” input from theoperator.

If the acquisition is not finished, such as when the ultrasound probe isstill actively acquiring ultrasound data according to the lungultrasound protocol and/or there are remaining views/imaging modes inthe lung ultrasound protocol, method 300 returns to 304 and continuesacquiring ultrasound data with the ultrasound probe according to thelung ultrasound protocol.

Once the acquisition is finished, such as responsive to the completionof the lung ultrasound protocol, method 300 proceeds to 320 and includesscoring each ultrasound image based on a percentage (or magnitude) ofpleural irregularities in the image. For example, the processor maydetermine the percentage of pleural locations that are identified asirregular relative to a total number of identified pleural locations. Asan example, the processor may count the healthy markers and theirregular markers in each annotated ultrasound image generated duringthe acquisition to quantify the percentage of pleural irregularities inthat image and score the image accordingly. For example, the processormay input the percentage into a look-up table stored in memory, whichmay output the corresponding score. As the percentage (or quantity) ofpleural irregularities increases, the score increases. In some examples,the processor may further take into account a number of identifiedB-lines in the image, with the score further increasing as the number ofidentified B-lines increases. For example, the processor may input boththe percentage of pleural irregularities and the number of identifiedB-lines into the look-up table to determine the score.

At 322, method 300 includes outputting the annotated ultrasound imagehaving the highest score to the display. The annotated ultrasound imagehaving the highest score may serve as a “best-frame” representation ofany lung pathology present and may be output immediately following theacquiring, for example. Thus, the annotated ultrasound image having thehighest score may be displayed in real-time during the acquisition andagain after the acquisition is completed. In this way, the annotatedultrasound image representing the greatest quantified irregularity isdisplayed to the operator in order to highlight any present lungpathology. When a panoramic view is displayed, the highest scoring framefor each rib space is selected and spliced with the other highestscoring frames, forming a composite image of portions of a plurality ofimages acquired at different times during the lung ultrasound protocol.

At 324, method 300 includes outputting a suggested diagnosis to thedisplay. As one example, the presence of B-lines and consolidation mayindicate an accumulation of fluid, such as due to bacterial or viralpneumonia (e.g., due to COVID-19). Further, the processor may take intoaccount a spread pattern of the irregularities among rib spaces (e.g.,concentrated in a few spots or spread across the lungs). As anotherexample, a lack of pleural sliding may indicate a collapsed lung. Assuch, the processor may compare the annotated ultrasound image havingthe highest score to a plurality of models corresponding to healthy lungor disease states and select the model having the best fit to output asthe suggested diagnosis. The suggested diagnosis may be output as atext-based message alongside or overlapping the displayed highestscoring annotated image. As one example, when the best fit model ispneumonia, the message may read, “Consider pneumonia.” In addition to oras an alternative to the suggested diagnosis, the processor may outputsuggestions on findings, such as irregular pleura, a sub-pleuralconsolidation, and the like.

At 326, method 300 optionally includes generating and outputting a 3Dlung model to the display. For example, the 3D lung model may begenerated when a lung sweep is performed during the lung ultrasoundprotocol. The 3D lung model may be patient-specific or may be generic.For example, the patient-specific lung model may be sized according tolength and width measurements determined during the lung sweep and usinga 3D probe sensor, as described above. As another example, thepatient-specific 3D lung model may be generated via image fusion with ahigh resolution-CT dataset, a chest x-ray image, or an MR image. The 3Dlung model may include one or more acquired ultrasound images overlaidon the model. As one example, the 3D lung model may include the highestscoring annotated lung ultrasound image for each view positioned at ananatomically relevant position with respect to the lungs. The 3D modelmay be rotatable via user input so that the operator may evaluatedifferent views. Further, sections of lung having pathologies may beindicated over the 3D model, such as via the annotations described aboveat 316 or via other annotations (e.g., text-based messages).

Turning briefly to FIG. 8, an example 3D lung model 800 is illustrated,which is output to a display 801. The display 801 may be the displaydevice 118 of FIG. 1, for example. The 3D lung model 800 includes awindpipe (e.g., trachea) 802, a left lung 804, and a right lung 806. Acropped panoramic lung ultrasound image 808 is overlaid on the left lung804. In the example shown, the cropped panoramic lung ultrasound image808 is contoured to the left lung 804 in order to show a more anatomicalrepresentation of the pleural spaces. Although only one panoramic lungultrasound image is shown in the example of FIG. 8, additional panoramiclung ultrasound images may be positioned on different locations of the3D lung model 800, such as on a back of the left lung 804, a side of theleft lung 804, a front of the right lung 806, a back of the right lung806, or a side of the right lung 806. Further, although not explicitlyshown in FIG. 8, the panoramic lung ultrasound image 808 may includeannotations, as described above.

Returning to FIG. 3, at 328, method 300 includes saving the unannotatedand annotated images to memory (e.g., the non-transitory memory 206 ofFIG. 2). Further, raw, unprocessed ultrasound data may be saved, atleast in some examples. The memory may be local to the ultrasoundimaging system or may be a remote memory. For example, the unannotatedand annotated images may be saved and/or archived (e.g., as a structuredreport in a PACS system) so that they may be retrieved and used togenerate an official, physician-signed report that may be included inthe patient's medical record (e.g., the EHR). Method 300 may then end.

In this way, the imaging system automatically identifies and quantifiespleural irregularities in images obtained via a lung ultrasoundprotocol. As a result, a mental burden on the operator may be decreased.Additionally, a variability between operators in pleural irregularitydetection accuracy and frequency is decreased. Overall, an accuracy of adiagnosis may be increased while an amount of time before the diagnosisis made may be decreased.

Next, FIGS. 6A and 6B show a first example sequence 600 of displayoutputs that may occur while acquiring lung ultrasound images (FIG. 6A)and upon completion of the acquisition (FIG. 6B). Looking first at FIG.6A, a series of lung ultrasound images are shown with respect to a timeaxis 602. For illustrative clarity, each lung ultrasound image iscropped to a location around a pleural line, and the same pleural lineis shown in each image (e.g., each image is acquired within the same ribspace). Each lung ultrasound image is displayed on a display 601, whichmay be the display device 118 of FIG. 1, for example, and includeshealthy markers 603 and irregular markers 605 overlaid at both the upperand lower boundaries of the imaged pleural line. The healthy markers 603are similar to the healthy markers 404 introduced in FIG. 4, and theirregular markers 605 are similar to the irregular markers 406 of FIG.4. Further, each displayed image includes a quantification of thepercentage of pleural irregularities in that image. It may be understoodthat although five lung ultrasound images are included in the sequence600, in other examples, more or fewer than five ultrasound images may beincluded.

A first lung ultrasound image 604 is acquired at a first time point t1,is analyzed to distinguish healthy pleura from irregular pleura (e.g.,according to method 300 of FIG. 3), and is output to the display 601 insubstantially real-time (e.g., substantially at time point t1). Thefirst lung ultrasound image 604 includes 85.2% irregular pleura. Asecond lung ultrasound image 606 is acquired, analyzed, and output tothe display 601 at a second time point t2, which occurs after the firsttime point t1, and has 80.8% irregular pleura. A third lung ultrasoundimage 608 is acquired, analyzed, and output to the display 601 at athird time point t3, which occurs after the second time point t2, andhas 55.6% irregular pleura. A fourth lung ultrasound image 610 that isacquired, analyzed, and output to the display 601 at a fourth time pointt4, which occurs after the third time point t3, also has 55.6% irregularpleura, although the upper and lower boundaries of the pleural line haveshifted slightly relative to the third lung ultrasound image 608. Afifth and final lung ultrasound image 612 is acquired, analyzed, andoutput to the display 601 at a fifth time point t5 and has 88.9%irregular pleura. Following the fifth time point t5, the acquisition isfinished.

Referring now to FIG. 6B, following completion of the acquisition, thefifth lung ultrasound image 612 is selected as the highest scoring imageframe because it displays the highest percentage of irregular pleura ofthe obtained images. Thus, the fifth lung ultrasound image 612 is againoutput to the display 601, while the first lung ultrasound image 604,the second lung ultrasound image 606, the third lung ultrasound image608, and the fourth lung ultrasound image 610 are not shown again on thedisplay 601 unless specifically selected by a user (e.g., operator).

FIGS. 7A and 7B show a second example sequence 700 of display outputsthat may occur while acquiring lung ultrasound images (FIG. 7A) and uponcompletion of the acquisition (FIG. 7B). Looking first at FIG. 7A, aseries of lung ultrasound images are shown with respect to a time axis702. For illustrative clarity, each lung ultrasound image is cropped toa location around a pleural line, and the same pleural line is shown ineach image (e.g., each image is acquired within the same rib space).Each lung ultrasound image is displayed on a display 701, which may bethe display device 118 of FIG. 1, for example, and includes healthymarkers 703 and irregular markers 705 overlaid at a lower boundary ofthe imaged pleural line. The healthy markers 703 are similar to thehealthy markers 404 introduced in FIG. 4, and the irregular markers 705are similar to the irregular markers 406 of FIG. 4. Further, eachdisplayed image includes an indication of the percentage of pleuralirregularities in that image. It may be understood that although fivelung ultrasound images are included in the sequence 700, in otherexamples, more or fewer than five ultrasound images may be included.

A first lung ultrasound image 704 is acquired at a first time point t1,is analyzed to distinguish healthy pleura from irregular pleura (e.g.,according to method 300 of FIG. 3), and is output to the display 701 insubstantially real-time (e.g., substantially at time point t1). Thefirst lung ultrasound image 704 includes 43.8% irregular pleura. Asecond lung ultrasound image 706 is acquired, analyzed, and output tothe display 701 at a second time point t2, which occurs after the firsttime point t1, and also has 43.8% irregular pleura, although the lowerboundary is positioned differently than in the first lung ultrasoundimage 704. A third lung ultrasound image 708 is acquired, analyzed, andoutput to the display 701 at a third time point t3, which occurs afterthe second time point t2, and has 81.3% irregular pleura. A fourth lungultrasound image 710 is acquired, analyzed, and output to the display701 at a fourth time point t4, which occurs after the third time pointt3, has 25.0% irregular pleura. A fifth and final lung ultrasound image712 is acquired, analyzed, and output to the display 701 at a fifth timepoint t5, and has 34.4% irregular pleura. Following the fifth time pointt5, the acquisition is finished.

Referring now to FIG. 7B, following completion of the acquisition, thethird lung ultrasound image 708 is selected as the highest scoring imageframe because it displays the highest percentage of irregular pleura ofthe obtained images. Thus, the third lung ultrasound image 708 is againoutput to the display 701, while the first lung ultrasound image 704,the second lung ultrasound image 706, the fourth lung ultrasound image710, and the fifth lung ultrasound image 712 are not shown again on thedisplay 701 unless specifically selected by a user (e.g., operator).

In this way, a processor may automatically identify and quantify pleuralirregularities by evaluating medical images from a patient using one oridentification and scoring algorithms. The processor may alert ahealthcare professional to the detected irregularities by annotating themedical images displayed on a display device in real-time as well asoutputting a suggested diagnosis and/or identified irregularities. As aresult, an amount of time the healthcare professional spends reviewingthe medical images may be reduced, enabling the healthcare professionalto focus on patient care and comfort. Further, a “best-frame”representation of the pleural irregularities may be selected via ascoring algorithm and displayed following the acquisition, in less thanreal-time. Further still, by including the best-frame overlaid on a 3Drendering of a lung model, the pleural irregularities may be displayedin an anatomically relevant environment in order to further simplify adiagnostic process.

A technical effect of automatically detecting pleural irregularities inmedical images is that an accuracy and frequency at which irregularitiesare detected may be increased.

In one embodiment, a method comprises: acquiring a series of medicalimages of a lung; identifying a pleural line in each medical image ofthe series; evaluating the pleural line for irregularities in eachmedical image of the series; and outputting an annotated version of eachmedical image of the series, the annotated version including visualmarkers for healthy pleura and irregular pleura. In a first example ofthe method, the pleural line is a substantially horizontal segment ofbrighter pixels, and identifying the pleural line in each medical imageof the series comprises: evaluating consecutive images of the series ofmedical images to determine an area having a highest amount of localchange between the consecutive images; and identifying an upper borderand a lower border of the pleural line within the determined area basedon a brightness change between pixels. In a second example of themethod, which optionally includes the first example, identifying theupper border and the lower border of the pleural line is via an edgedetection or gradient change algorithm. In a third example of themethod, which optionally includes one or both of the first example andthe second example, identifying the pleural line in each medical imageof the series comprises identifying an upper border and a lower borderof the pleural line based on a brightness change between pixels, andevaluating the pleural line for the irregularities comprises:determining an irregularity score for each pixel between the upperborder and the lower border of the pleural line; characterizing a givenpixel as healthy responsive to the irregularity score being less than athreshold score; and characterizing the given pixel as irregularresponsive to the irregularity score being greater than the thresholdscore. In a fourth example of the method, which optionally includes oneor more or each of the first through third examples, the irregularityscore is a product of a first score and a second score for the givenpixel. In a fifth example of the method, which optionally includes oneor more or each of the first through fourth examples, the first score isdetermined based on a vertical gap of the pleural line at a horizontallocation of the given pixel, the first score increasing as the verticalgap increases. In a sixth example of the method, which optionallyincludes one or more or each of the first through fifth examples, thesecond score is determined based on a brightness of the given pixelrelative to neighboring pixels, the second score increasing as thebrightness of the given pixel decreases relative to the neighboringpixels. In a seventh example of the method, which optionally includesone or more or each of the first through sixth examples, the visualmarkers for the healthy pleura and the irregular pleura include a firstvisual marker positioned at each location of the pleural line havingpixels characterized as healthy and a second visual marker positioned ateach location of the pleural line having pixels characterized asirregular. In an eighth example of the method, which optionally includesone or more or each of the first through seventh examples, the firstvisual marker includes one or more of a different shape, color, and sizethan the second visual marker, and one or both of the first visualmarker and the second visual marker is positioned along the upper borderand/or the lower border of the pleural line. In a ninth example of themethod, which optionally includes one or more or each of the firstthrough eighth examples, outputting the annotated version of eachmedical image of the series comprises: outputting the annotated versionof each medical image of the series in real-time during the acquiring;and outputting the annotated version of one selected medical image ofthe series immediately following the acquiring, the one selected medicalimage of the series having a greatest amount of the irregular pleurarelative to the healthy pleura. In a tenth example of the method, whichoptionally includes one or more or each of the first through ninthexamples, acquiring the series of medical images of the lung includesacquiring at least one of ultrasound imaging data, magnetic resonanceimaging data, computed tomography data, x-ray data, and positronemission tomography data.

In another embodiment, a method comprises: generating a plurality oflung ultrasound images while acquiring ultrasonic signals according to aprotocol; visually indicating irregular pleura in each of the pluralityof lung ultrasound images on a display in real-time during theacquiring; and upon completion of the protocol, selecting one imagehaving a greatest relative amount of the irregular pleura from theplurality of lung ultrasound images and outputting only the one image tothe display. In a first example of the method, visually indicating theirregular pleura in each of the plurality of lung ultrasound images onthe display in real-time during the acquiring comprises: identifyingborders of a pleural line in each of the plurality of lung ultrasoundimages based on at least pixel brightness; determining an irregularityscore for each location of the pleural line; and visually distinguishinglocations of the pleural line having irregularity scores less than athreshold from locations of the pleural line having irregularity scoresgreater than or equal to the threshold. In a second example of themethod, which optionally includes the first example, the irregularpleura comprise the locations of the pleural line having theirregularity scores greater than or equal to the threshold, anddetermining the irregularity score for each location of the pleural linecomprises: determining a first score based on a vertical gap betweenpleura in each location; determining a second score based on a pixeldimness of pleura in each location relative to neighboring locations;and determining the irregularity score as a product of the first scoreand the second score. In a third example of the method, which optionallyincludes one or both of the first and second examples, each of theplurality of lung ultrasound images comprises a panoramic lungultrasound image including a plurality of rib spaces, and selecting theone image comprises: selecting separate images having the greatestrelative amount of the irregular pleura for each rib space of theplurality of rib spaces; and generating the one image as a compositeimage using portions of the separate images selected for each rib spaceof the plurality of rib spaces. A fourth example of the methodoptionally includes one or more or each of the first through thirdexamples and further comprises overlaying the one image on athree-dimensional lung model that is output to the display.

In yet another embodiment, a system comprises: an ultrasound probe; adisplay device; and a processor configured with instructions innon-transitory memory that, when executed, cause the processor to:acquire ultrasound data via the ultrasound probe according to a lungimaging protocol; generate a plurality of images from the ultrasounddata; evaluate each of the plurality of images to detect pleuralirregularities; and output a visual indication of the pleuralirregularities on the display device in real-time. In a first example ofthe system, the processor is further configured with instructions in thenon-transitory memory that, when executed, cause the processor to:quantify the pleural irregularities in each of the plurality of images;select one of the plurality of images having a highest quantification ofthe pleural irregularities; and output the selected one of the pluralityof images to the display device responsive to the lung imaging protocolcompleting. In a second example of the system, which optionally includesthe first example, the processor is further configured with instructionsin the non-transitory memory that, when executed, cause the processorto: determine a suggested diagnosis based on the highest quantificationof the pleural irregularities and a spacing of the pleuralirregularities in the selected one of the plurality of images; andoutput the suggested diagnosis to the display device. In a third exampleof the system, which optionally includes one or both of the first andsecond examples, the lung imaging protocol includes a lung sweep and theultrasound probe includes a three-dimensional sensor, and the processoris further configured with instructions in the non-transitory memorythat, when executed, cause the processor to: generate athree-dimensional lung model based on a length of the lung sweep and awidth measured by the three-dimensional sensor; and output the selectedone of the plurality of images overlaid on the three-dimensional lungmodel to the display device.

As used herein, an element or step recited in the singular and proceededwith the word “a” or “an” should be understood as not excluding pluralof said elements or steps, unless such exclusion is explicitly stated.Furthermore, references to “one embodiment” of the present invention arenot intended to be interpreted as excluding the existence of additionalembodiments that also incorporate the recited features. Moreover, unlessexplicitly stated to the contrary, embodiments “comprising,”“including,” or “having” an element or a plurality of elements having aparticular property may include additional such elements not having thatproperty. The terms “including” and “in which” are used as theplain-language equivalents of the respective terms “comprising” and“wherein.” Moreover, the terms “first,” “second,” and “third,” etc. areused merely as labels, and are not intended to impose numericalrequirements or a particular positional order on their objects.

Embodiments of the present disclosure shown in the drawings anddescribed above are example embodiments only and are not intended tolimit the scope of the appended claims, including any equivalents asincluded within the scope of the claims. Various modifications arepossible and will be readily apparent to the skilled person in the art.It is intended that any combination of non-mutually exclusive featuresdescribed herein are within the scope of the present invention. That is,features of the described embodiments can be combined with anyappropriate aspect described above and optional features of any oneaspect can be combined with any other appropriate aspect. Similarly,features set forth in dependent claims can be combined with non-mutuallyexclusive features of other dependent claims, particularly where thedependent claims depend on the same independent claim. Single claimdependencies may have been used as practice in some jurisdictionsrequire them, but this should not be taken to mean that the features inthe dependent claims are mutually exclusive.

1. A method, comprising: acquiring a series of medical images of a lung;identifying a pleural line in each medical image of the series;evaluating the pleural line for irregularities in each medical image ofthe series; and outputting an annotated version of each medical image ofthe series, the annotated version including visual markers for healthypleura and irregular pleura.
 2. The method of claim 1, wherein thepleural line is a substantially horizontal segment of brighter pixels,and identifying the pleural line in each medical image of the seriescomprises: evaluating consecutive images of the series of medical imagesto determine an area having a highest amount of local change between theconsecutive images; and identifying an upper border and a lower borderof the pleural line within the determined area based on a brightnesschange between pixels.
 3. The method of claim 2, wherein identifying theupper border and the lower border of the pleural line is via an edgedetection or gradient change algorithm.
 4. The method of claim 1,wherein identifying the pleural line in each medical image of the seriescomprises identifying an upper border and a lower border of the pleuralline based on a brightness change between pixels, and wherein evaluatingthe pleural line for the irregularities comprises: determining anirregularity score for each pixel between the upper border and the lowerborder of the pleural line; characterizing a given pixel as healthyresponsive to the irregularity score being less than a threshold score;and characterizing the given pixel as irregular responsive to theirregularity score being greater than the threshold score.
 5. The methodof claim 4, wherein the irregularity score is a product of a first scoreand a second score for the given pixel.
 6. The method of claim 5,wherein the first score is determined based on a vertical gap of thepleural line at a horizontal location of the given pixel, the firstscore increasing as the vertical gap increases.
 7. The method of claim5, wherein the second score is determined based on a brightness of thegiven pixel relative to neighboring pixels, the second score increasingas the brightness of the given pixel decreases relative to theneighboring pixels.
 8. The method of claim 4, wherein the visual markersfor the healthy pleura and the irregular pleura include a first visualmarker positioned at each location of the pleural line having pixelscharacterized as healthy and a second visual marker positioned at eachlocation of the pleural line having pixels characterized as irregular.9. The method of claim 8, wherein the first visual marker includes oneor more of a different shape, color, and size than the second visualmarker, and wherein one or both of the first visual marker and thesecond visual marker is positioned along the upper border and/or thelower border of the pleural line.
 10. The method of claim 1, whereinoutputting the annotated version of each medical image of the seriescomprises: outputting the annotated version of each medical image of theseries in real-time during the acquiring; and outputting the annotatedversion of one selected medical image of the series immediatelyfollowing the acquiring, the one selected medical image of the serieshaving a greatest amount of the irregular pleura relative to the healthypleura.
 11. The method of claim 1, wherein acquiring the series ofmedical images of the lung includes acquiring at least one of ultrasoundimaging data, magnetic resonance imaging data, computed tomography data,x-ray data, and positron emission tomography data.
 12. A method,comprising: generating a plurality of lung ultrasound images whileacquiring ultrasonic signals according to a protocol; visuallyindicating irregular pleura in each of the plurality of lung ultrasoundimages on a display in real-time during the acquiring; and uponcompletion of the protocol, selecting one image having a greatestrelative amount of the irregular pleura from the plurality of lungultrasound images and outputting only the one image to the display. 13.The method of claim 12, wherein visually indicating the irregular pleurain each of the plurality of lung ultrasound images on the display inreal-time during the acquiring comprises: identifying borders of apleural line in each of the plurality of lung ultrasound images based onat least pixel brightness; determining an irregularity score for eachlocation of the pleural line; and visually distinguishing locations ofthe pleural line having irregularity scores less than a threshold fromlocations of the pleural line having irregularity scores greater than orequal to the threshold.
 14. The method of claim 13, wherein theirregular pleura comprise the locations of the pleural line having theirregularity scores greater than or equal to the threshold, anddetermining the irregularity score for each location of the pleural linecomprises: determining a first score based on a vertical gap betweenpleura in each location; determining a second score based on a pixeldimness of pleura in each location relative to neighboring locations;and determining the irregularity score as a product of the first scoreand the second score.
 15. The method of claim 12, wherein each of theplurality of lung ultrasound images comprises a panoramic lungultrasound image including a plurality of rib spaces, and selecting theone image comprises: selecting separate images having the greatestrelative amount of the irregular pleura for each rib space of theplurality of rib spaces; and generating the one image as a compositeimage using portions of the separate images selected for each rib spaceof the plurality of rib spaces.
 16. The method of claim 12, furthercomprising overlaying the one image on a three-dimensional lung modelthat is output to the display.
 17. A system, comprising: an ultrasoundprobe; a display device; and a processor configured with instructions innon-transitory memory that, when executed, cause the processor to:acquire ultrasound data via the ultrasound probe according to a lungimaging protocol; generate a plurality of images from the ultrasounddata; evaluate each of the plurality of images to detect pleuralirregularities; and output a visual indication of the pleuralirregularities on the display device in real-time.
 18. The system ofclaim 17, wherein the processor is further configured with instructionsin the non-transitory memory that, when executed, cause the processorto: quantify the pleural irregularities in each of the plurality ofimages; select one of the plurality of images having a highestquantification of the pleural irregularities; and output the selectedone of the plurality of images to the display device responsive to thelung imaging protocol completing.
 19. The system of claim 18, whereinthe processor is further configured with instructions in thenon-transitory memory that, when executed, cause the processor to:determine a suggested diagnosis based on the highest quantification ofthe pleural irregularities and a spacing of the pleural irregularitiesin the selected one of the plurality of images; and output the suggesteddiagnosis to the display device.
 20. The system of claim 18, wherein thelung imaging protocol includes a lung sweep and the ultrasound probeincludes a three-dimensional sensor, and the processor is furtherconfigured with instructions in the non-transitory memory that, whenexecuted, cause the processor to: generate a three-dimensional lungmodel based on a length of the lung sweep and a width measured by thethree-dimensional sensor; and output the selected one of the pluralityof images overlaid on the three-dimensional lung model to the displaydevice.